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SYSTEMS  RESEARCH  GROUP 
A.  Charnes,  Director 


The  convex  duality  theory  as  developed  by  Charnes-Cooper-Kortanek 
which  is  valid  for  non-differentiable  quasi- concave  functions  is  applied  to 
derive  a  gradient  inequality  for  a  non-differentiable  function  involving  a 
symmetric  quadratic  form  with  positivity  required  only  on  a  given  convex 
polyhedral  cone,  generalizing  a  theorem  of  Edmund  Eisenberg.  No  assump¬ 
tion  is  needed  either  that  the  primal  problem  attain  its  extremal  value,  since 
the  CCK  duality  theory  includes  this  possibility.  The  theory  directly  applies 
to  the  non-homogeneous  problem  where  now  the  given  cone  is  replaced  by  the 
intersection  of  a  finite  number  of  half  spaces. 


If  f(x)  is  a  convex  differentiable  function  over  Rn  and  A  is  a  real 

T 

mxn  matrix,  then  the  gradient  inequality  (x-x  )  f  i  0  whenever  Ax  £0 

O  X 

o 

holds  if  and  only  if  xq  minimizes  f(x)  over  X  =  {x|Ax  ^  0}  .  This  inequality 

has  a  Farkas-Minkowski  equivalent  which  follows  directly  from  the  Farkas- 

Minkowski  lemma.  This  dual  e  ^uivalent  is  exactly  the  statement  of  convex 

duality  as  developed  by  Charnes-Cooper-Kortanek—^  which  is  also  valid  for 

non-differentiable  convex  functions.  It  is  this  statement  of  convex  duality 

2/  3/ 

that  E.  Eisenberg  —  and  S.  M.  Sinha  —  developed  in  an  elegant  manner  for 

T  t  ~ 

the  important  special  case,  f{x)  =  a  x  +  (x  Cx)2  . 


1 J  See  [5],  pp.  605-608. 

2/  See  [10]. 

See  [*?].  This  development  parallels  the  methods  of  Eisenberg's  earlier 
article,  "Supports  of  a  Convex  Function,"  Bulletin  of  the  AMS,  Vol.  68, 
No.  3,  May  1962,  pp.  192-195,  to  prove  a  dual  theorem  tor  a  quadratic 

$(x)  =  a^x  +  2  (xT C.x)2  , 

i=l  1 


where  C.  are  as  above. 
1 
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Part  of  their  concern  (e.  g. ,  Sinha)  has  been  that  one  cannot  directly  apply 
the  usual  convex  programming  methods  to  the  problem, 

min  a^ x  +  (x^  Cx)2  with  Ax  ^  b  ,  and  x  0 

because  of  non-differentiability  of  (x  Cx)2  .  However,  this  type  of  non¬ 
differentiability  can  be  overcome,  as  well  as  that  of  the  more  general 

i  t  i  A 

non- differentiable  constraints  A  x  +  (x  B  x)2  -  b  ,  by  a  simple  transformation 
and  the  introduction  of  nspacing"  variables.  This  device  of  Charnes  and 
Cooper  may  be  found,  for  example,  in  Charnes -Cooper's  "Deterministic 
Equivalents  for  Optimizing  and  Satisficing  under  Chance  Constraints," 
Operations  Research,  Vol.  11,  No,  1,  pp.  18-39,  January-February  1963, 

(which  was  circulated  as  an  ONR  report  in  1960-61)  or  in  the  Charnes-Cooper- 
Thompson  paper,  "Characterizations  by  Chance- Constrained  Programming" 
at  the  Symposium  of  Mathematical  Programming,  June  1962,  whose  proceed¬ 
ings  were  published  as  Recent  Advances  in  Mathematical  Programming, 

R.  L.  Graves  and  P.  Y/olfe,  editors. 

In  particular  the  extended  dual  equivalent  of  the  "gradient"  inequality 
for  the  non- differentiable  symmetric  function,  f(x) ,  of  Eisenberg  will  now  be 
directly  derived  from  the  CCK  duality  theory,  in  a  manner  which  generalizes 
Eisenberg's  theorem.  — ^ 

T  T  ~ 

Theorem  Assume  that  f(x)  =  a  x  +  (x  Cx)2  ^  0  whenever 

T 

x  c  X  =  {x|Ax  ^  0}  ,  where  C  is  symmetric  and  x  Cx  2  0  on  X . 

Then  there  exist  tt  £  0  and  z  such  that 
(i)  Az  £  0 

(ii)  vF. A  +  a^  +  z^ C  =  0  and  z^Cz  ^  1 . 

is  a 


1/  See  E.  Eisenberg,  [10],  Theorem  3;  0'»r  theorem,  however, 
generalization  since  we  do  not  require  x  Cx  ^  (a  x)  . 
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Proof  Consider  the  problem: 

(I) 

T 

min  a  x  +  t 

subject  to  -Ax  £  0 

T  2 

-x1  Cx  +  i  £  0 

t  2  0  . 

Assume  for  the  moment  that  the  constraint  set  is  convex,  which  would  be  the 

T  2 

case,  for  example,  if  -x  Cx  +  t  were  quasi- concave.  It  should  be  empha¬ 
sized  that  in  convex  duality  theory,  concave  constraint  functions  are  not  needed 
per  se,  but  only  that  the  constraint  set  which  they  determine  be  convex. 
Generalizations  to  quasi- concave  constraint  functions  were  developed  by 

Arrow-Hurwicz-Uzawa  and  by  Arrow-Enthoven.  — ^ 

T  2 

In  our  formulation  the  set  determined  by  -x  Cx  +  t  £  0  alone  is  not 
convex.  However,  the  additional  inequality  t  £  0  chooses  one  nappe  of  the 
elliptic  hyperboloid  determined  by  the  quadratic  inequality,  thus  defining  a 
convex  set  whose  differentiable  support  system  is  now  determined  by  usual 
processes  of  differentiation.  Observe  that  the  infimum  exists,  and  in  fact 
a  minimum  value  of  zero  is  actually  attained.  Introducing  this  support  system 
brings  problem  (I)  into  the  following  equivalent  inequality  form. 

(I) 

T 

mm  ax  +  t 
subject  to  -Ax  £  0 

x^(-Cx  )  +  t  £  0  for  all  x  with  x^Cx  =  1,  -Ax  £  0  — ^ 

a  a  a  a  a 

t  *  0 

J_/  See  [  l]  and  [  2] . 

T  t  ~ 

2/  Since  the  cone  a  x  +  (x  Cx)2^  0  contains  -Ax  £  0  by  hypothesis,  qne  may 
as  we  have  done,  further  restrict  xa  to  range  over  -7Lxa£  0.  If  xTCx  =  0 
on  Ax  S  0  ,  then  the  ordinary  Farkas- Minkowski  lemma  applies. 
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Lemma  1  The  linear  inequality  system  for  (I)  as  presented  above  completely 

T  2 

determines  the  convex  constraint  set,  -Ax  ^0,-xCx  +  t  £0,  t  £  0  . 

T  2 

Proof  Over  x(X  and  t  2:  0  ,  the  inequality  -x  Cx  +  t  £  0  determines 

T  2 

a  convex  set,  and  therefore  for  any  (x  ,  t  )  with  x  Cx  =  t  and  non- 

J  at  a  at  at  a 

vanishing  gradient,  (-2Cxff,  2t^)  ^  0  ,  it  follows  that 

x^{-Cx  )  +  tt  £  0  .  — ^ 
at  at 

Therefore,  if  (x,  t)  is  feasible  for  problem  (I),  it  satisfies  the  system 

xT(-Cx  )  +  t  S  0  for  all  x  with  xTCx  =  1 
at  a  a  a 

t  S  0  . 

T  2 

Conversely,  if  (x,  t)  satisfies  this  system  and  -Ax  S  0  ,  then  -x  Cx  +  t  S  0  ; 

T  X 

for  in  the  trivial  case  of  x  Cx  =  0  ,  we  are  done.  In  the  case  of  x  Cx  ^  0, 

x  X 

set  x  = — — - r  80  that  x  Cx  =  1  and  -Ax  S  0  ,  and  therefore  x  is  a 

*  (xTCx)>  *  * 

member  of  our  indexing  set.  Therefore, 

+  t  S  0  =>  t  S  (xT Cx)2  =>  t^  S  x"  Cx  .  Q.  E.  D. 

Lemma  2  The  linear  inequality  system  for  (I)  is  a  Farkas- Minkowski 
system. 

Proof  Since  problem  (I)  has  finite  optima  it  suffices  to  consider  bounded 
coefficients  because  it  is  only  these  which  the  differential  system  need 
encompass,  i.  e.  ,  it  suffices  to  consider  a  compact  index  set,  I,  contained 
in  {x  lx  Cx  =  1  ,  Ax  ^  0}  .  Therefore  the  possibly  infinite  differential 

1_/  Compare  [2],  p.  788,  where  the  property  of  quasi- concavity  of  the 

constraint  functions,  gJ  ,  is  used  to  establish  this  fact.  They  point  out 
that  for  this  purpose  it  suffices  that  the  constraint  set  be  convex. 


I  I 
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subsystem  has  interior  points.  We  now  distinguish  two  cases  on  the  remaining 
finite  subsystem,  -Ax  ^  0  . 

Case  1  There  exists  x  such  that  -Ax  >0.  It  then  follows  that  (x  ,t  ) 

■  —  —  o  o  o  o 

is  an  interior  point  for  the  entire  system  for  tQ  sufficiently  large.  Thus,  we 
have  a  canonically  closed  Haar  system  and  therefore  a  Farkas- Minkowski 
system. 

Case  2  The  system  Ax  S  0  has  no  interior  points,  i.  e. ,  X  =  {xlAx  =  0} 
and  X  is  an  (n-r)  flat,  1  ^  r  ^  m  .  In  this  case  we  may  choose  r  linearly 
independent  equations  from  this  subsystem  which  also  determine  X  •  say 
AjXj+  0  »  where  A^  is  rxr  and  nonsingular  and  A^  is  rx(n-r)  . 

Therefore,  problem  (I)  becomes 


min 

T  T 

alxl  +  a2x2  +t 

subject  to 

Alxl  +  A2x2  =  0 

xlT(-Clx«>+x2(-C2xa>  +  t*0  >  xa‘  1 

where  C^  is  rxn,C£if  (n-r)xn  and 


'2j 


t  *  0 

=  C  . 


To  prove  the  Farkas -Minkowski  property  for  the  given  system  we  first 
reduce  variables  via  x^  =  -A^A2*2  to  attain: 


«r> 


,  T  T.-l.  . 
min  (a2  -  A^  A^x^ 


+  t 


T  t  -1  T 

subject  to  x_(-C_x  +  A,  (A  )  C.x  )+t^0,  x  (I 
J  2  2a  21  a  a 

t  *  0 


1/  See  [12]  for  a  general  discussion  and  proof  of  a  similar  technique  for 
resolving  duality  gaps  for  infinite  linear  inequality  systems  which  do  not 
have  interior  points. 
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Since  the  original  system  is  compact,  the  reduced  system  also  is  and,  in 
addition,  contains  no  trivial  inequalities. 

Clearly  the  linear  inequality  system  of  (Ir)  has  interior  points. 
Therefore,  it  is  a  Haar  system  and  the  semi- infinite  duality  theorem  applies 
to  (If)  and  its  dual  (11^. ): 


max  2  0*  \ 


+  On 


subject  to 


2( 

or 


T  JT  ,  T  _T  A-1A  u 
■x  C,  +  x  C,  A,  A.)X 
a  2  a  1  1  2  a 


=  a. 


T  A-1A 
al  A1  A2 


+  n  =  l 


1  ,  u 
a  r 


i  0 


Therefore,  let  (x*,t*  ;  X.*,  p*)  he  a  dual  optimal  solution.  In  order  to  go  from 

T  t  T  -1  ^ 

the  reduced  system  (Ir)  to  the  original  system  (I),  simply  set  x^  =  -x|  A2  (Aj  ) 

and  <o*T  =  a?A/+  Zx^C^A’^X#  .  Therefore  (xf ,  xS,  t#  ;  <*>*,  X.*,  p*)  is  a  dual 
i  i  at  1  l  or  i  c 

optimal  solution  for  (I)  as  we  now  easily  check.  First,  the  dual  problem  to 
(I)  is  of  course: 

(II) 

max  wT.  0  +  Z  0  •  \  +  0*  p 

* 

T 

subject  to  <oTA,  +  E  {-xTC.T)X.  s  al 

i  a  at  L  at 

T .  ,  v  ,  T^T..  _  T 

W  A2+  «  *  « C2  "  2 

2  X  +  p  =  1 

a  <*  r 
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T 

To  check  dual  feasibility  for  (II),  observe  first  that  the  expression  for  is 

T 

verified  directly  by  substitution  for  to#  and  X* .  The  required  expression 
T  T 

for  a^  is  obtained  by  substituting  for  w*  into  the  first  constraint  of  (11^ ) ,  i.  e.  , 
a J  =  Ja\1Az  *  r  (-xJ  cj  +  xj  C*  A-/ V  -  X  xj  C*  X*  . 


Finally  the  equality  of  the  (I)-  (II)  functionals  follows  directly  from  the 
equality  of  the  (1^ )- (U^)  functionals.  Therefore  the  proof  of  Lemma  2  is 
complete.  Thus,  because  of  Lemma  2,  there  is  no  loss  of  generality  if  we 
assume  that  Ax  S  0  has  interior  points;  a  Farkas -Minkowski  system  is 
always  attained,  and  therefore  the  semi-infinite  duality  theory  applies  to  the 
dual  problems  (I)  and  (II).  For  future  reference  in  the  proof  of  our  theorem, 
we  rewrite  problem  (II)  in  its  original  form. 


(II) 


max  w  •  0  +  2  0*  \ 


+  p  •  0 


subject  to  -<*?* A  +  £(-x^C)X 


=  a 


SX 
a  « 


+  H  =  1 


T  T  T 

where  <*>  ,  X  ,  p  £  0  and  \  have  only  finitely  many  non-zero  components. 


Now  the  minimum  of  problem  (I)  is  assumed  at  x*  =  0  ,  t*  =  0,  and 

possibly  at  other  points  with  t*  t  0  .  Assume,  in  general,  that  (x*,t#)  is 

an  optimal  solution  to  (I).  A  dual  optimal  solution  exists  by  the  extended  dual 

theorem,  say,  ,  p*) .  Now  since  OcX  and  EX^+p^sl,  if  follows 

a  a 


z  =  2x*  X*  +  0*  p*«  X  (i.  e. ,  feasible)  since  all 
a  a  a 

x*’s  are  feasible. 
a 


that 
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Therefore,  {-2x*X*r  Cxp+t*S  0  for  all  x^  with  Cx^  =  1,  -A::^  £  0. 

rp 

In  particular,  \*  \*(2x*X*)  Cx*  ,  and  summing  we  get 


t^  SK*t*S  (Sx*\*)TC(Ix*X*,  ,  i.e., 
a  a  a  a  a) 

zT  Cz  S  t* 


This  inequality  is  with  respect  to  any  optimal  solution  to  problem  (I),  (x*,t*). 

Any  non-degenerate  optimal  solution,  i.e.,  x*^Cx*  ^  0  yields  an  equivalent 
>9c  T 

one  with  t  =  1 ,  namely,  a  i  +1  =  0.  Thus ,  the  inequality 

(x*  Cx*)* 

T 

becomes  z  Cz  £  1  ,  which  of  course  already  includes  the  case  of  degenerate 
solutions . 

Therefore,  the  z  that  occurs  in  the  optimal  dual  solution  is  exactly 
the  one  required  of  the  theorem,  i.  e.  , 

zTCz  JS  1 

a^  +  z^  C  +  uFa  -  0  and  Az  £  0  . 


To  complete  the  proof  of  the  theorem  we  consider  the  general  case 

T  2 

where  the  constraint  set  Ax  £  0  ,  -x  Cx  +  t  £  0  •  t  £  0  may  not  be  convex  if 

T 

all  that  we  require  of  C  is  that  it  be  symmetric  and  x  Cx  St  0  on  Ax  £  0  . 

2 

For  example,  the  set  given  by:  t  -  x^x^  £  0  ,  t  £  0  ,  and  x^,  x^  *  0  is  not 
convex.  However,  we  may  still  apply  our  duality  theory. 

Consider  the  partial  support  system  given  by 

-Ax  *  0 

x^(-Cx  )  +  t  £  0  for  x  «  I  , 
a  a 


t  *  0 
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where  I  is  defined  as  follows: 


I  =  {x  I  C  x  =  1  ,  Ax  S  0 }  (1  {x  I  II  x  II  ^  k  } 

or  a  a  or 

T 

for  some  fixed  (large)  k  >  0  .  Thus,  I  is  a  compact  set  in  R  .  Since  x  Cx 
is  continuous,  it  is  bounded  on  I,  and  the  image  is  compact.  Further,  the 
infinite  subsystem  has  interior  points,  as  seen  by  simply  taking  some 
T 

t  >  max  lxACx  I  .  Therefore  by  Lemma  2  we  again  obtain  a  Farkas- 
orcl  a  a 

Minkowski  system. 

To  show  that  this  minimization  problem  is  bounded  below,  we  need 
only  show  that  the  dual  problem  is  feasible.  Following  Eisenberg,  — ^we  now 
show  that  the  equality  system. 


T 

TT 


(-A)  +  zT(-C)  =  aT 
Az  +  y  =  0  ; 


v,  y 


> 


0 


is  consistent.  By  the  finite  Farkas-Minkowski  theorem  this  is  equivalent  to 
T 

showing  that  a  u  -  0  whenever  -Au  £  0 

-Cu  +  A^v  =  0 

v^O  , 

T 

To  this  end,  assume  (u,  v)  satisfies  the  above  system.  Claim  u  Cu  £  0  ; 

T  T  T  T  T 

for  u  Cu  =  u  A  v  =  v  Au  ,  and  v  Au  2  0  since  Au  ^  0  and  v  i  0  .  On  the  other 

hand,  Au  ^  0  =>  Cu  S  0 ;  hence  u^Cu  =  0  .  Thus  Au  3  0  =>  a^u  +  (u^ Cu)*  ^  0  , 

T 

which  finally  implies  a  u  £  0  .  Observe  that  we  have  used  all  the  assumptions 
of  the  theorem. 


1_/  See  E.  Eisenberg  [lO],  Appendix  B,  where  a  similar  argument  is  presented 
for  dual  feasibility. 
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Thus,  the  dual  problem  is  feasible,  and  by  taking  k  large  enough  we 
can  include  enough  points  in  the  index  set  I  so  that  the  partial  support  system 
contains  the  proper  coefficients  which  permit  dual  feasibility.  This  completes 
the  proof  of  the  theorem  as  stated. 

In  closing,  we  remark  that  the  general  convex  duality  theory  also  pro¬ 
vides  analogous  results  for  the  related  non- homogeneous  problem  — ^  where 
now  the  feasibility  region  is  given  by  Ax  ^  b  .  In  this  general  context,  the 
minimum  need  not  be  assumed,  which  is  consistent  with  the  Extended  Dual 

Theorem.  However,  to  avoid  this  situation  in  the  statement  of  duality, 

2/ 

Sinha  —  introduces  an  assumption  which  requires  the  existence  of  a  feasible 

T 

solution  (-rr,  z)  with  z  Cz  <  1  if  the  dual  problem  is  feasible.  This  assumption 

is  not  necessary  for  duality  theory  because  the  extended  dual  theorem  includes 

3/ 

the  possibility  of  the  primal  problem  not  attaining  its  extremal  value.  —  To 

illustrate  this  phenomenon  consider  the  following  example  introduced  by 
4/ 

Sinha—  and  its  semi-infinite  equivalent. 


1 J  See  [lO],  section  III. 

2/  See  [9].  p.  15,  for  an  exact  statement  of  this  assumption. 
3/  For  an  example  of  this  situation  see  [8],  p.  215. 

4/  See  Sinha,  ibid.,  p.15. 
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In  this  example,  inf(-X£  +  t)  =  0  ;  for  clearly  t-X£-  0  over  the  constraint 
set,  and  taking  x^=2,  x^  =  n,  and  t  =  n  +  2/n  for  n  =  l,  2  .  . , ,  proves  the 
assertion.  Note  that  this  infimum  is  not  assumed.  Introducing  supports, 
(I)  becomes: 


(I) 


min 


+  t 


subject  to  Xj 


£  2 


x^(-x^)  +  x^t-x®)  +  t  £  0  for  all  (x^,  x^)  with  (x^)^+  (x^)^ 1. 


1/ 


The  dual  is: 


t  *  0 


(II) 


max 
subject  to 


2w 

u>+  E(-x?U  =0 

a  1  a 

Z(-x*)X  =  -1 

a  2  * 

S  X  +  UL  =  1 

a  a  r 


where  w,  X,  p  £  0  . 

Now,  taking  x^*=  0  ,  =  1 ,  X*  =  1 ,  p*  =  0  ,  w*  =  0  and  remaining 

variables  zero,  we  attain  the  dual  optimum.  Observe  that  in  this  example 

the  z  of  the  inhomogeneous  counterpart  to  Theorem  1  is  (0,1),  so  that 
T 

z  Cz  =  1  . 

1_/  Observe  that  for  the  inhomogeneous  problem,  one  may  no  longer  further 
restrict  by  Ax^^  b  as  in  the  homogeneous  case. 
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